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One presents methodology and algorithms to prepare a causal system in order to achieve desired 
performances if only input-output data are known and when no other informations are available. 
This can be done with mean of evolutionnary programming and set inversion methods, such as 
OA t/i- algorithm or SIvIA. 
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The Holy Grail of Science and Engineering is the ability to perform adjustments of a system in order to get the 
^^ desired performances. For example, how to settle a robot in order to make it moving toward the desired target, or 
which materials properties are well-suited for a defined functionnality. Thus, many problems are dealing with the 
1^^ inversion of the relation between adjustments and performances. 

■^ Some powerful methods based on interval arithmetic have been developed those last years, such as SIvlJ^^ (Set Inver- 

^7- sion Via Interval Arithmetic) and ip- a lgorit hnP^ (Probabilistic Set Inversion) which is inspired from SIvIA. They are 
^^ both based on interval arithmeticP^'^, which has been introduced originally to compute very quickly and in a reliable 
tG way quantities represented with intervals. Nowadays, interval arithmetic has been extended to become a computing 
j^ framework which permits to perform error analysis by computing ma t hema tic bounds, and to solve problems such 
Cj as non linear problems, partial differential equations, and set inversiorP^'i^. It finds a large place of applications in 

I I material sciences, controllability, automatism, robotics, embedded systems, biomedical, haptic interfaces, form opti- 
mization, analysis of architecture plans, ... 
Cn We are interested to compute the adjustments needed by a system to achieve specific performances, when the mod- 

elization of the system is unknown and when only experimental and measured data are available. In order to perform 
such kind of difficult set inversion, one proposes in this paper to combine SIvIA-\ike algorithms such as ?/;-algorithm 
Jp^ with some evolutionnary schemes, in particular symbolic regression using genetic programming. In the next section, 
f""*^ one recalls briefly the basis of the ^-algorithm, and exhibit its limitations. Thereafter, one presents symbolic regres- 
^-Z sion based on genetic programming to simulate the system that we would like to invert. We exhibit in the third 
(—» K section some examples with numerical applications developed in ruhy^ which is an elegant, powerful object oriented 

pg programming language. 
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d Let us note TZ C M" the set of adjustments, and V <zW the set of performance of a system. The mathematical 

modelization of our problem consists of the computation oiS = f~^{V)mZ, as shown on figure (ll]), where / : M" — > Rp 

is the function giving performances from adjustments. For semantical reasons, one has to perform this set inversion 

within the interval semi-group lrf!3. But unfortunately, in many cases, this function is not known, i.e no formulas can 

be given to reproduce the output values from the input ones. However some input-output measurements, meaning 

adjustments-performances experimental or empirical values can be available and that's the reason one proposes to 

use first the symbolic regression based on genetic programming to approximate this function (figure pi) in the sense 

of uniform convergence on the adjustments set, and it will be used afterward within a set inversion scheme such as 

pure SIvIA or the -^-algorithm (figure ^ . 
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Figure 1: Set Inversion consists to compute 5 = / ''{V) n 7?. for a defined function / with mean of interval arithmetics. 

III. SYMBOLIC REGRESSION VIA GENETIC PROGRAMMING (SRFGP) 

A. Scheme 

Usually if some input-output data are known, one performs function parameters fitting using optimization methods. 
Because it needs some knowledge of an empirical and parametrized form of the function, and since it is hard for the 
numerical scheme not to be trapped in a local minimum, it can be rather useful and powerful to perform a symbolic 
regression in order to find a function converging uniformly to the unknown one as shown on figure ([2]). 
Symbolic regression is a good candidate to achieve such kind of procedures. It has the great advantage to give a 
formal expression of the wanted function which converges uniformely on the adjustment set (figure^ to the unknown 
one. Uniform convergence means that it is stronger than pointwise convergence and that the speed of convergence is 
not point dependant. 

Symbolic regression can be done in different ways. In this article, we focus on the so-called Symbolic Regression via 
Genetic Programming (GPj^^^^SRvGP). GP was initially developed for inductive automatic programming and is 
well-suited for symbolic regression, controller design, and machine learning tasks. One can consider GP algorithms 
as an extension of the well-known genetics algorithms. GP's scope is to use induction to devise a computer program. 
This is achieved by using evolutionary operators on candidate programs with a tree structure to improve the adaptive 
fit between the population of candidate programs and an objective function. An assessment of a candidate solution 
involves its execution. Symbolic regression with input-output data is achieved just by considering a function as a 
computer program, which is represented with a tree (figure pi) and to use those data in the objective function. 
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Figure 2: Symbohc Regression via Genetic Programming permits to build an approximation of an unknown function defined 
on 7?., and valued on V. 



Let us suppose that some input-output data (TZoi'Po) have been measured. Therefore, we would like to find out 
a symbolic formula which can describe the system by mean of the symbolic regression. However, it can be useful 
sometimes to perform pre-treatements with Fourier or wavelets transforms^'^ (PRE-treatment on figure (l3|) . This signal 
processing needs that one has to choose the frequencies which have to been kept or removed. On the other hand, it 
can have meaning to find a solution which takes into account the noise, but this is another approach which is not 
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Figure 3: Combining Genetic Programming (GP) and Set Inversion (SI) permits first to find an approximation of tlie model and 
in a second step to invert performance set in order to find the efficient and possible adjustments. Experimental input-output 
data could be filtered and decimated if it is needed in order to remove some frequencies and to avoid overfitting phenomena. 

discussed in this paper and can be studied in another framework. In a second step, a decimation of the signal points 
can be useful to decrease the computational time and to avoid overfitting phenomena. Wavelet analysis seems to be a 
promising tooP^lto achieve that. Finally, one obtains a new input-output data set which can be used for the symbolic 
regression of the system (figure [3| . 

B. Numerical example of SRvGP 

For example, we propose to illustrate this procedure on data produced with the function 

fix) = sin(5a;) • e""'' for x G [-3, 3] (1) 

This signal is very interesting, not only because it is a wavelelp2, but because it has several local extrema and fast 

oscillations. 

We have added to this function a random uniformly distributed noise e(x) G [—0.25,0.25] for x E [—3,3] in order to 

see if, in this case, the genetic programming scheme is sensible to small perturbations or could extract the right signal 

we are looking for. 



Original and noised signal sin(5x)*exp(-x'^2) 
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Figure 4: Performance set for the function f{x) = sin(5 ■ x) ■ e ^ . An uniform random noise e{x) G [—0.25, 0.25] for x G [—3, 3] 
has been added to the initial output data set f([-3,3]). 

The symbolic regression has been implemented within a ruby script and a MPI library for this language that we 
have developed for the scope of this present article. The code is well-suited for parallel computer, launching several 
jobs on different nodes with different random seeds. After convergence of each population, a last run is done on a 
single node involving the best candidates of each previous nodes within a larger population. 

The final result obtained in less than 3 minutes for this example with 16 processors and for a cost function about 
10~^, is given in a EBNI^^ (Extended Backus-Naur Form) expression (* (* (sin (* x 5)) l)(exp (* (neg 1)(* x x)))) 
and can be represented by the tree shown on figure ([5]). This expression represents the function 



f{x) — sin(5a;) • e ^ 



(2) 



We have choosen a grow method for the tree developpement with a maximal depth of 5 and a set of function such 
as {+, — , 'i /i exp, log, sin, cos, tan, neg} where neg : x i— )■ —x. Figure Q shows the noised signals for adjustments on 
[—3, 3]. It is interesting that in this example the signal filtering has not been used in this example since the genetic 
program absorbs the noise and finds the right solution. Thus, the symbolic regression finds finally the right function 
/ = / on the adjustments set [—3, 3] even in presence of small noise and without filtering. 




Figure 5: Result for the genetic programming of the f{x) = sin(5a;) • e ^ noised example. This tree corresponds to the 
approximation of /, f{x) = sin(5a;) • e~^ = f{x) on [—3,3] with function basis set { + ,—,-,/, exp, log, sin, cos, tan, neg}. The 



IV. PROBABILISTIC SET INVERSION 

A. Scheme 

Let us assume now that the model describing the adjustment-performance is known. In order to perform set inversion 
we propose to use the f/^-algorithnP^ (Probabilistic Set Inversion) inspired from SIvIA^iSl but without boolean tests 
and with a conditional probability calculation and domain bissections. The interval arithmetics used is based on 
free algebraa^^and permits to define univoquely inclusion function associated to the natural one and to avoid data 
dependencie9^'2II3, This yields to accepted or rejected domains only. In this algorithm, one computes the following 
conditional probability with notations related to the figure (IT]), 



p{x) = p([f](x) c V I fix) e [f](x), Vx e x) 

^ mes{[f]{X)nV) 
mes{[f]{X)) 
mes{y n V) mes{I) 



mes{y) mes(y) 



(3) 



where mes is the Lebesgue measure in W (p-dimension volume). 
If the image of an adjustement subset X is included in the performance set "P, then I = y. This yields the probability 
to be equal to f and one adds the subset X to the solution list. If the image set y is not included in the performance 



set, then mes{I) = and the probabihty equals zero, therefore the subset X is rejected and removed from the 
Hst of interval candidates, li p{X) G ]0, 1[, then X is bissected in all directions and i/)-algorithm apphes the same 
procedure recursively for the resulting intervals until the size is lower than a fixed volume resolution of the intervals 
or until the sets are accepted or rejected. Moreover, -0-aIgorithm uses a well-defined interval arithmetic based on free 
algebraS-SJl,, it guarantees to get well-defined inclusion functions and avoids in particular data dependancie a^ - ^ - ^^ - ^'l . 



B. Numerical examples of 7/;-algorithm applications 



1. One dimensional examp 



Let's suppose now that the model of our problem is the previous function found in the equation (pi) with a SRuGP 
scheme. Now one would like to solve the following inequation : 



-\<Kx)<^, for a: €[-2,2] 
This can be rewritten as a set inversion for / with TZ = [—2, 2] and V = [—\,-^] 



(4) 



PSI for sin(5x)*exp{-x'~2) with R=[-2,2] and P=[-0.25,0.5] 
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Figure 6: Result of set inversion with i/)-algorithni for f{x) = sin(5a;) • e ^ with 7?. = [—2, 2] and V 
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2. Higher dimensional examples 



Let us give an other example with the function / 



fix, y)^{x^ -y^ ■e'' + x- e^, x ■ {x + y) - y"^) 



(5) 



One would like to invert this function for TZ = [—5,5]^ and V = [—5,5]^. This corresponds to the resolution of a 
strongly non-linear inequations system. The set inversion presented on figure ([7]) shows clearly that the ■i/'-algorithm 
implemented in the algebraic arithmetic we use is not data dependant. The variables appear more than once in the 
formal expression of the function /. This shows how 5'/wL4-inspired methods such as ^-algorithm are powerful and 
efficient to inverse performance set through a continuous function. The CPU time for this inversion is about 130 
seconds on a two processors computer (1.67 Ghz Intel) for a spatial surface resolution of 10^"'. 

There is no limitation for the dimensions of the adjustments and performances sets as shown on figure ((sl) for 
the function / : R^ — > M^. Due to the bissection of the domain, the computational time remains still exponen- 
tial. However , it is possible to decrease the computational time constant with mean of parallclization using domain 
decompositiorPSHZl, The adjustement set is divided on the first axis, and each processor performs the ^/'-algorithm on 
one of those subdomains. The master processor collects all the results at the end of the calculations. 
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PSI for f(x,y)={x"2-y'^2*exp(x)+x*exp(y),x*(x+y)-y"2) 
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Figure 7: f/i-algorithm for f{x, y) = {x'^ — y^ ■ e' ~\- x ■ e" ,x ■ {x + y) — j/^) with TZ — P — [—5, 5]^ 



PSI for f{x,y,z)=(x,y,z,x*x-y*y+z*z) 
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Figure 8: i/j-algorithm for f{x,y,z) = {x,y,z, 



z^) with 7^ = [-5, 5]^ and V = [-10, 10]'' 



V. CONCLUSION AND PERSPECTIVES 



The paper has presented a new methodology for adjustments of a system in order to satisfy desired performances. 
This technique, combining genetic programming for symbohc regression with set inversion, only assumes that input- 
output data are known. The approach is decomposed into two steps: the symbolic regression and the set inversion. 
The role of the symbolic regression is to find an accurate model for the system by fitting all input-output data. 
The main advantage of symbolic regression compared to more classical fitting techniques such as splines, wavelet, 
etc. is that the resulting model has a short symbolic expression which is essential to make interval set inversion 
method efficient. The set inversion with -^-algorithm is able to solve many interesting problems occuring in science 
and engineering such as strongly non-linear inequations and equations, and optimization problems. 
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